1. Field of the Invention
The present invention relates to methods and apparatus for displaying computer-resident data, such as graphical information for display on a computer monitor. More specifically, the present invention relates to the writing of display data into a pool of arbitrarily selected tiles for subsequent ordering and rendering onto a display. When operated in conjunction with a view position sensing apparatus, display of multiple objects, such as windows, as a function of viewing position is provided. It is therefore possible to display objects such as windows, which are typically displayed in a static, two dimensional display space, in a simulated three dimensional, dynamic display space. That is, the present invention facilitates providing a viewer with a relative view or parallax perspective of the displayed objects.
2. Description of the Prior Art
There is presently a problem associated with displaying objects on a computer display device (generally referred to herein as a monitor) that the physical area within which the objects are displayable is limited by the surface area of the display portion of the display device. For example, the target area or active pixel portion ("screen")of cathode ray tubes, liquid crystal or plasma panels or the like employed in computer monitors have a finite width (x-dimension) and height (y-dimension), and objects to be displayed by such devices must be displayed within the dimensions of the screen.
While efforts have been made to simulate depth (z-dimension) of the display to provide a three-dimensional display space, in most computing applications objects are most commonly displayed in two dimensions. For example, display windows found in modern windowing systems are presented to the user as if directly on the surface of the monitor, with virtually no depth. Similar limitations are found in computer aided design (CAD) programs, illustration programs, page layout programs, etc.
There is a further problem with display methods and devices in that most often they do not account for the position of the user with respect to the display device. For example, while a sophisticated computer animation program is capable of synthesizing a three-dimensional view of an object by shading, highlighting, cropping, and other visual cues, such programs assume a fixed position of the viewer with respect to the display device. This limitation means that in displaying objects--even three--dimensionally rendered objects--there is no accommodation for the relative position of those objects with respect to the viewer. For example, while overlapping windows are known in the art, that part of a window overlapped by another window is clipped, and a change in viewing position does not change the portion of the window which is clipped. The same holds true for more dynamic environments such as video games, etc. The relative position of objects with respect to the viewer provides significant depth cues; specifically of interest in the present case is the "parallax" of the displayed objects.
Parallax occurs when simultaneously viewing at least two objects which are each at a different distance from the viewer. Each object is viewed in a line of sight. Motion of the viewer relative to the objects changes the line of sight and produces a new perspective of the objects. As the line of sight changes, the two objects appear to have changed position relative to one another. This effect is called parallax.
In the physical world, the degree to which two objects appear to have moved relative to one another when viewed at two different viewing positions is a function of their respective distances from the viewer. For example, as the viewer moves, the object closer to the view appears to move more than the object farther from the viewer. In general, the greater the distance between the objects, the greater the visual effect. These relative changes in position are one type of visual cue providing information about the positions, sizes, motion, etc. of viewed objects.
I have discovered that a user's perception of the three-dimensionality of displayed computer-generated images is greatly enhanced when parallax shift of the displayed objects is provided. In addition, it is possible to "extend" the viewing area of a display beyond the physical boundaries of the monitor by providing a parallax effect; in a sense allowing the user to peer around the edges of the display into a virtual display space. These techniques rely on a system of software and hardware forming an aspect of this invention which departs from the tradition serial method of writing to display buffers and reading out the display buffers scan line-by-scan line. The present invention may employ novel hardware and software for tracking the position from which the display is viewed by a user.
Typical systems for displaying computer-based data include a graphics engine which interprets commands from a microprocessor. The graphics engine generates pixel data which is written to a frame buffer. This frame buffer is essentially a display address bit map; each address in the frame buffer stores pixel data for a single pixel of the display. Most often, the frame buffer is organized to correspond in size and bit sequence to a scan line of the monitor The data is loaded from the frame buffer to a shift register. The data is then clocked out by the shift register to a color palette chip, and ultimately drives the display of a scan line on the monitor.
The process associated with such a system is bit-mapped in nature. That is, the pixels are written to and read from the frame buffer in an order advantageous for serial read-out to the display. The pixel ordering in the frame buffer has no explicit regard for their association with the object which they represent nor the distinction between objects. The nature of this strictly bit-mapped process becomes disadvantageously slow, and in many cases impracticable, when the process is called upon to display a dynamically selected view of data, for example, when panning an environment in a so-called virtual reality (VR) application.
Such systems are referred to herein as pixel-oriented, because the highest discernible level of data organization in the frame buffer is a pixel. The present invention is an alternative approach to the pixel-oriented system. Referred to as an object-oriented system, the highest discernible level of data in the frame buffer is an object. That is, the pixels are written to and read from the frame buffer in a way that preserves the association of pixels to an object. This association is advantageous for positioning of objects to render a dynamically selected view of data. Object-oriented frame buffers for a dynamically selected view of the data are not known in the art.
In order to provide such a parallax perspective, it must be possible to determine information regarding the user's viewing position and the use of that information to tailor the display accordingly. A number of techniques are known to track the position of the viewer, and display objects as a function of viewing position. Examples of apparatus developed for tracking viewing position include head-tracking goggles, glasses or helmets. These display devices are worn by a user and usually include control over the projection of different images to each of the user's left and right eyes to simulate a stereoscopic display. Motion of these devices relative to some fixed point as determined by appropriate means usually controls the display of objects, for example providing relative motion as between two displayed objects.
One example of these methods and devices include U.S. Pat. No. ('179) to Waldern, which discloses wearing a helmet outfitted with one video display unit for each eye, the motion of the helmet tracked to provide a visualized virtual model. Another example is U.S. Pat. No. ('055) to Smith, which discusses using video glasses to allow a user's head position to determine which portion of a panorama is displayed to the viewer.
However, there are numerous problems with these devices. For example: the hardware and software resources needed to operate these devices are expensive and complex; in stereoscopic display systems, they require the recording and/or rendering of two separate images, one for each eye; the image updates lag substantially behind viewer movements resulting in motion sickness of the user; they provide inferior display quality either by reducing the animation rate or the display resolution; etc. For these reasons, special purpose, expensive display systems have been the only means for determining viewing position information and using that information when generating a display of data.
In addition, known systems providing view position sensing have required stereoscopic displays where a separate image is generated for each eye. The lack of an autoscopic (single image) display with viewing position tracking has meant that applications running on relatively inexpensive, desktop computer systems or low-end graphics workstations have not been able to employ either meaningful view position sensing or the parallax effect.
The lack of practical, low cost view position-sensitive display capability precludes the application of the parallax effect to routine computer applications such as drawing programs, word processing and spreadsheets, etc. Examples of areas where there is a need for parallax display include those applications where there are a large number of objects to be displayed simultaneously, or where the size of the objects to be displayed require a display area greater than that of the screen. Such is the case, for example, in computer-aided design systems, where multiple "windows" of varying size need to be displayed, preferably simultaneously. Another example comes from the art of video games where there is often a desire to simultaneously display multiple objects such as opponents, background, etc., or to provide the player with the impression that she is actually in a room or other environment in which the game is being played. Parallax perspective will benefit any application in which a three-dimensional display space is being synthesized.